Skip to content

fix: dont optimize out benches #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 4, 2025
Merged

fix: dont optimize out benches #18

merged 1 commit into from
Jul 4, 2025

Conversation

not-matthias
Copy link
Member

No description provided.

@codspeed-hq CodSpeed HQ
Copy link

codspeed-hq bot commented Jul 3, 2025 •
edited
Loading

CodSpeed Instrumentation Performance Report

Merging #18 will degrade performances by 98.91%

Comparing cod-1096-cpp-benchmarks-are-optimized-out (835fcda) with main (719c41f)

Summary

⚡ 2 improvements
❌ 20 (👁 20) regressions
✅ 20 untouched benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
👁 BM_Capture[int_test] 91.4 ns 121.4 ns -24.71%
👁 BM_memcpy[4096] 92.8 ns 4,424.7 ns -97.9%
👁 BM_memcpy[512] 92.8 ns 944.7 ns -90.18%
👁 BM_memcpy[64] 92.8 ns 334.4 ns -72.26%
👁 BM_memcpy[8192] 92.8 ns 8,335.8 ns -98.89%
👁 BM_memcpy[8] 92.8 ns 305.6 ns -69.64%
👁 BM_rand_vector 91.4 ns 182.8 ns -50%
👁 BarTest[MyFixture] 90.3 ns 121.7 ns -25.8%
👁 DoubleTest[MyTemplatedFixture, double] 90.3 ns 121.7 ns -25.8%
👁 FooTest[MyFixture] 90.3 ns 121.7 ns -25.8%
👁 IntTest[MyTemplatedFixture, int] 90.3 ns 121.7 ns -25.8%
👁 TestA[MyTemplate1, int] 90.3 ns 121.7 ns -25.8%
👁 TestB[MyTemplate2, int, double] 90.3 ns 121.7 ns -25.8%
⚡ BM_Capture[int_test] 119.4 ns 91.9 ns +29.91%
👁 BM_memcpy[4096] 90.3 ns 4,394.2 ns -97.95%
👁 BM_memcpy[512] 90.3 ns 972.5 ns -90.72%
👁 BM_memcpy[64] 90.3 ns 362.2 ns -75.08%
👁 BM_memcpy[8192] 90.3 ns 8,305.3 ns -98.91%
👁 BM_memcpy[8] 90.3 ns 304.2 ns -70.32%
👁 BM_rand_vector 90.3 ns 153.1 ns -41.02%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@codspeed-hq CodSpeed HQ
Copy link

codspeed-hq bot commented Jul 3, 2025 •
edited
Loading

CodSpeed WallTime Performance Report

Merging #18 will degrade performances by 100%

Comparing cod-1096-cpp-benchmarks-are-optimized-out (835fcda) with main (719c41f)

Summary

❌ 22 (👁 10) regressions
✅ 20 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
❌ BarTest[MyFixture] < 1 ns < 1 ns -100%
❌ DoubleTest[MyTemplatedFixture, double] < 1 ns < 1 ns -100%
❌ FooTest[MyFixture] < 1 ns < 1 ns -100%
❌ IntTest[MyTemplatedFixture, int] < 1 ns < 1 ns -100%
❌ TestA[MyTemplate1, int] < 1 ns < 1 ns -100%
❌ TestB[MyTemplate2, int, double] < 1 ns < 1 ns -100%
❌ BM_Capture[int_string_test] < 1 ns < 1 ns -100%
👁 BM_Capture[int_test] < 1 ns < 1 ns -100%
👁 BM_rand_vector < 1 ns 2.2 ns -100%
❌ BM_Template1_Capture[int_string_test] < 1 ns < 1 ns -100%
❌ BM_Template1_Capture[two_type_test, int, double] < 1 ns < 1 ns -100%
👁 BarTest[MyFixture] < 1 ns < 1 ns -100%
👁 DoubleTest[MyTemplatedFixture, double] < 1 ns < 1 ns -100%
👁 FooTest[MyFixture] < 1 ns < 1 ns -100%
👁 IntTest[MyTemplatedFixture, int] < 1 ns < 1 ns -100%
👁 TestA[MyTemplate1, int] < 1 ns < 1 ns -100%
👁 TestB[MyTemplate2, int, double] < 1 ns < 1 ns -100%
❌ BM_Capture[int_string_test] < 1 ns < 1 ns -100%
❌ BM_Capture[int_test] < 1 ns < 1 ns -100%
👁 BM_rand_vector < 1 ns 1.7 ns -100%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@not-matthias not-matthias force-pushed the cod-1096-cpp-benchmarks-are-optimized-out branch 2 times, most recently from 7e537b6 to 6377dd1 Compare July 4, 2025 09:44
@not-matthias not-matthias force-pushed the cod-1096-cpp-benchmarks-are-optimized-out branch from 6377dd1 to 65710a6 Compare July 4, 2025 09:49
@not-matthias not-matthias force-pushed the cod-1096-cpp-benchmarks-are-optimized-out branch from 65710a6 to 835fcda Compare July 4, 2025 09:50
@not-matthias not-matthias merged commit 835fcda into main Jul 4, 2025
12 checks passed
@not-matthias not-matthias deleted the cod-1096-cpp-benchmarks-are-optimized-out branch July 4, 2025 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@art049 art049 art049 approved these changes

@GuillaumeLagrange GuillaumeLagrange Awaiting requested review from GuillaumeLagrange

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@not-matthias @art049